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INTRODUCTION 

Many  factors  occur  during  the  life  of  a  government  sponsored 
research  and  development  project  which  could  have  a  direct  bearing 
upon  the  quality  of  the  final  product.   Looking  at  a  project  chron- 
ologically, however,  there  are  a  number  of  decisions  which  occur 
prior  to  the  time  that  work  actually  begins  which  might  have  an 
impact  on  final  performance. 

This  report  will  deal  with  the  direct  impact  on  total  perform- 
ance of  certain  decisions  occurring  prior  to  the  start  of  the  pro- 
ject.  On  the  government  side,  decisions  as  to  the  type  of  contract 
(sole  source  or  competitive)  and  constraints  on  the  choice  of  sub- 
contractors will  be  examined.   Within  the  company,  the  focus  will  be 
on  the  criteria  for  selection  of  project  personnel,  the  percent  of 
total  effort  subcontracted,  and  certain  characteristics  of  initial 
cost  and  schedule  estimates. 

Focusing  on  initial  conditions  does  not  imply  that  other  factors, 
whose  time  frame  spans  the  entire  project,  are  unimportant.   Nor  does 
it  eliminate  the  possibility  that  an  initial  decision  might  affect 

final  perfonnance  indirectly  by  influencing  the  contractor's  decision 

2 
as  to  the  best  form  of  project  to  adopt.    The  rationale  is  that  many 


Only  a  fraction  of  the  data  collected  during  the  past  three  years  in 
a  broad  study  of  project  management  will  be  discussed.   Participating 
in  this  investigation  as  research  associates  and  assistants  have  been 
J.  Randall  Brown,  Michael  Gold,  Arthur  J.  Hansen,  Kenneth  R.  Hootnick, 
Richard  B.  Maffei,  James  E.  Mahoney,  Robert  L.  Pearson,  Jean-Paul 
Richard,  and  David  Straight  Jr. 

2 

See  Marquis  and  Straight  (1965)    for  a  comparison  of  the  effectiveness 

of  different  forms  of  project  organization. 
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of  the  problems  which  arise  during  the  life  of  a  project  are  less 
subject  to  control  by  both  the  government  and  the  contractor  than 
the  initial  decisions  to  be  examined  in  this  paper. 

Research  Methods 

A  project,  carried  out  by  a  company  under  contract  with  a  govern- 
ment agency,  has  many  advantages  as  a  unit  of  study  in  the  investi- 
gation of  effective  performance  of  research  and  development.   Such  a 
project  has  a  defined  beginning  and  end,  it  has  Initial  cost  and 
schedule  estimates  as  well  as  a  work  statement  of  technical  requirements, 
and  it  has  separate  and  audited  financial  accounting. 

Projects  were  selected  on  the  basis  of  three  criteria: 

1.  All  were  R&D  contracts  awarded  by  a  government  agency 
(or  industry  prime  in  several  cases)  to  an  industrial 
firm. 

2.  All  were  over  $1  million  in  total  value  (excluding 
follow-on  production  work) . 

3.  All  were  very  recently  completed  or  rapidly  nearing 
completion. 

Projects  were  located  in  two  ways„   Alternate  firms  in  the  list 
of  100  largest  defense  and  space  firms  were  invited  to  cooperate,  and 
if  they  were  willing,  a  division  laboratory  was  randomly  selected,  and 
its  most  recently  completed  (or  terminated)  project  became  the  target 
of  study.   The  other  source  was  a  government  contracting  agency,  in 
which  one  or  more  projects  were  chosen  randomly  from  those  most  re- 
cently completed. 
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Information  on  each  project  was  obtained  from  five  sources:   the 
laboratory  manager  (L^0  ,  the  project  manager  (PM) ,  the  government 
technical  monitor  (GTM) ,  the  government  contract  administrator  (GCA) , 
and  the  company  contract  administrator  (CA) .   Judgments  of  successful 
performance  and  a  statement  of  their  criteria  for  evaluating  successful 
performance  were  obtained  from  the  first  four.   The  laboratory  and 
project  manager  described  the  organization  and  operations  of  the  pro- 
ject group.   They  and  the  technical  monitor  also  provided  information 
on  success  in  meeting  schedule  deadlines  and  cost  criteria.   Admin- 
istrative performance  was  described  by  the  contract  administrators 
in  both  the  company  and  the  government  agency.   The  questionnaires 
used  in  this  study  were  developed  on  the  basis  of  published  literature 
and  of  interviews  with  experienced  project  managers,  and  were  pretested 
on  approximately  20  projects. 

Characteristics  of  the  Projects 

Although  a  total  of  48  projects  were  studied,  inevitable  diffi- 
culties in  securing  some  of  the  desired  information  resulted  in  in- 
complete records  in  several  cases.   In  75%  of  the  cases  the  records 
are  complete  (LM,  CA,  PM,  GTM,  and  GCA) .   Incomplete  information 
results  primarily  from  an  inability  to  interview  either  the  LM,  GTM, 
or  the  GCA. 


See  Marquis  and  Straight,  (1965)    for  further  details. 


The  projects  studied  were  funded  by  12  government  agencies,  five 
in  the  Air  Force,  tv/o  in  the  Navy,  two  in  the  Army,  two  in  NASA  and 
one  in  another  civilian  agency.   The  32  firms  performing  the  projects 
are  large  corporations  in  tlie  aerospace  and  electronics  industries, 
26  are  among  the  100  largest  performers  of  government-funded  R&D. 
Five  more  are  smaller  but  in  the  same  technology.   The  other  one  is 
a  large  corporation  in  a  basic  industry. 

The  laboratories  or  engineering  facilities  in  which  the  projects 
were  performed  are  located  in  all  regions  of  the  country.   Four  are 
in  the  New  England  region,  13  in  the  Middle  Atlantic,  six  in  the  South 
and  Southv;est,  four  in  the  Middle  West  and  ten  on  the  West  Coast, 

The  laboratories  are  primarily  engaged  in  government  R&D  work, 
with  23  of  them  reporting  that  more  than  90  percent  of  their  work  was 
on  government  contracts,  and  only  tv/o  reporting  less  than  40  percent 
on  government  R&D. 

The  projects  ranged  in  size  from  $1  million  to  $60  million  with 
a  median  of  $4  million.   The  average  project  duration  v/as  3.4  years 
and  none  lasted  more  than  6  years.    Almost  all  of  the  projects  re- 
quired advances  in  the  "state  of  the  art"  in  a  technological  field 
such  as  advanced  radar  systems,  microminiaturization  of  electronics 
modules,  electronic  data  processing  interfaces  v/ith  telemetry  systems, 
etc.   The  projects  studied  are  more  developmental  than  fundamental 
research. 


Measures  of  Project  Performance 

It  is  presently  impossible  to  compare  the  technical  performance 
of  different  projects  by  any  objective  measures.   In  one  instance, 
speed  may  be  the  primary  technical  objective  of  a  system  (missile, 
airplane,  etc.).  while  in  another  case  range  is  most  critical.   In 
an  electronics  system  reliability  or  maintainability  may  be  the  chief 
goal.   Consequently  the  measures  of  successful  technical  performance 
used  in  this  study  are  expert  judgments  by  the  most  fully  informed 
individuals  (cf.  Peck  and  Scherer,  1962). 

Success  ratings  were  obtained  independently  from  the  project  man- 
ager, the  laboratory  manager,  the  government  contract  administrator 
and  the  technical  monitor.   Tlie  ratings  were  on  a  scale  from  one  to 
nine,  with  one  representing  an  outstanding  success  and  nine  signifying 
a  failure  (in  some  relative  sense,  since  no  project  in  this  study 
achieved  the  absolute  failure  of  being  cancelled  before  completion) . 
All  except  one  of  the  ratings  fell  in  the  range  of  one  to  five,  within 
which  there  was  a  symmetrical  distribution  with  the  mode  at  three. 

Tlie  measure  of  performance  used  in  this  study  is  the  average  of 
the  ratings  provided  by  the  project  manager,  the  government  technical 
monitor,  plus  a  constant  term  to  account  for  the  fact  that  project 
managers  were  consistently  more  optimistic  than  technical  monitors, 
Table  I  contains  the  resulting  distribution  of  technical  performance 
scores.   The  higher  the  absolute  score,  the  less  well  the  project  did 
technically. 


Information  on  cost  and  schedule  changes  was  gathered  from  the  project 
(Footnote  cont'd  on  next  page.) 
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TABLE  I 
DISTRIBUTION  OF  TECHNICAL  PERFORMANCE  SCORES  (N=48) 
Technical  Performance  Scorel  Frequency 

1.2  5 

1.7  2 

1.8  4 

1.9  2 

2.2  4 
2.4  2 
2.8  1 
3.6  9 
4.1  1 

4.3  7 
4.6  6 
5.0  3 
5.6  1 

^•^  MEDIAN  =  3.6  "'■ 


Footnote  cont'd  from  page  5. 

manager  and  contract  administrator  in  the  company  and  from  the  government 
contract  administrator.   Careful  cross-checking  on  the  reasons  for  contract 
changes  made  it  possible  to  eliminate  those  caused  by  a  customer's  action, 
such  as  a  change  in  scope  or  specifications,  or  the  unavailability  of  gov- 
ernment test  equipment  or  facilities.   Thus  only  those  slippages  which  v/ere 
directly  attributable  to  the  actions  of  the  contractor  firm  are  includec'. 
While  the  data  do  not  at  present  include  the  amount  of  cost  and  schedule 
variances,  only  those  greater  than  10  percent  are  included.   On  this  basis 
there  were  28  overruns  and  13  on  target  or  underruns .   These  results  are 
not  unlike  those  reported  by  Marshall  and  Heckling  (1962),  and  Peck  and 
Scherer  (1962).   The  rank  correlation  between  cost  and  schedule  perform.ance 
was  0.82,  so  the  two  are  combined  to  form  a  single  performance  measure. 
For  the  sample  of  projects  under  investigation,  technical  performance  is 
unrelated  to  cost  and  schedule  performance. 

Technical  performance  is  not  significantly  related  to  the  dollar  size  of 
the  contract,  nature  of  end  product  or  %  basic,  applied  or  development 
research  necessary  to  accomplish  objectives.   It  is  also  unrelated  to 
whether  the  project  was  considered  a  crash  program  (K=20)  or  not  (K=2C). 
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RESULTS 

Type  of  Contract 

One  of  the  initial  decisions  made  by  the  government  (buyer)  in 

the  procurement  process  concerns  the  number  of  firms  that  will  be 

requested  to  bid.   If  only  one  company  has  the  capability  necessary 

to  meet  the  technical  requirements,  the  contract  can  be  awarded  as 

a  sole  source  procurement.   Otherwise  some  form  of  competitive  process 

is  initiated  and  the  contract  is  awarded  to  the  company  that  wins 
this  competition. 

Of  the  A3  projects  for  which  data  are  available  28  awards  re- 
sulted from  some  form  of  competitive  procedure,  and  15  from  sole 
source  procurement.   Sole  source  contracts  have  a  mean  performance 
score  of  2.80  while  the  rest  liave  a  mean  performance  score  of  3.80. 
In  other  words,  sole  source  contractors  do  significantly  better 

technically  than  contractors  who  were  selected  b^  some  form  of  com- 

3 
petitive  procedure.    In  addition,  if  one  accepts  any  performance 

score  belovj  the  median  (2.8  or  lower)  as  being  highly  successful, 
(  see  Table  I)  53%  of  sole  source  contracts  would  be  judged  as  highly 
successful  while  only  28%  of  the  remainder  would  fall  into  this  cate- 
gory. 


See  Roberts  (1964)    for  a  comparison  between  the  formal  process  and 

the  actual  process  by  which  the  government  buys  research. 

2 

Mann-Whitney  U-Test,  p^.04 

3 

It  should  be  remembered  that  the  ranking  scheme  used  to  measure  per- 
formance leads  to  a  situation  where  the  lower  the  absolute  score  the 
better  the  project  was   judged   to  be   technically. 
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An  attempt  was  made  with  the  data  available  to  explain  why  a 
contract  was  awarded  as  sole  source.   No  relationships  were  found 
between  type  of  contract  and  such  factors  as  the  number  of  years 
a  laboratory  spent  in  the  technologies  required  by  the  project  or 
the  laboratory's  prior  experience  relative  to  others  in  the  field. 
(The  latter  was  measured  by  asking  the  laboratory  manager  for  his 
subjective  estimate  and  more  than  75%  of  all  respondents  said  they 
had  rraich  more  experience  than  others  in  the  field.')   Data  were  not 
available  concerning  the  factors  considered  by  the  government  in 
their  decision  to  contract  sole  source  in  the  first  place.   The 
available  data  do  not  permit  an  analysis  of  the  weight  given  to  the 
technical  aspects  of  the  competitive  proposals  relative  to  other 
factors . 

Subcontract  Activity 

The  next  decision  examined  concerned  the  percent  (on  a  dollar 
basis)  of  total  project  effort  which  was  subcontracted  outside  of 
the  organization.   Any  work  done  in  other  parts  of  the  same  organ- 
ization (e.g.  another  laboratory)  or  the  purchase  of  off-the-shelf 
hardware  items  was  not  included. 

Referring  to  Table  II,  it  can  be  seen  that  in  8  projects,  no 
work  was  subcontracted  while  in  2  projects  more  than  50%  of  the  ef- 
fort was  subcontracted.   The  median  amount  subcontracted  was  15  per- 
cent. When  percent  effort  subcontracted  is  correlated  with  technical 
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TABLE   II 


DISTRIBUTION   OF  %   OF   SUBCONTRACTED 

(EXCLUDING   THE    PURCHASE   OF 
OFF-THE-SHELF  HARDWARE) 


%   Subcontracted  Frequency 

8 
9 
11 
5 
6 
2 


00% 

^00 

<10% 

>10 

<20% 

^20 

<30% 

^30 

<50% 

>50 

% 
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performance,  it  is  found  that  the  more  work  done  outside,  the  less 
well  the  project  did  technically. 

Allen  (  1966  )  has  found  that  utilization  of  outside  vs.  Inside 
sources  of  information  in  the  performance  of  research  contracts  is 
associated  with  lower  rated  technical  solutions.   This  does  not  mean 
that  outside  sources  (e.g.  consultants)  cause  poor  performance.   More 
likely,  as  Allen  points  out,  it  probably  reflects  the  fact  that  the 
organization  lacked  the  technical  capability  to  do  the  job  in  the 
first  place.   Granted,  in  our  sample,  hardvs^are  of  some  sort  rather 
than  ideas  is  the  commodity  being  sought  from  outside  sources.   How- 
ever, it  seems  reasonable  to  assume  that  a  similar  lack  of  in-house 
capability  leads  an  organization  to  subcontract  and  for  reasons  as 

yet  unexplained,  such  behavior  is  strongly  associated  with  poorer 

2 
technical  performance. 

The  next  factor  examined  within  the  area  of  subcontract  activ- 
ity was  the  nature  of  the  restrictions  placed  upon  a  contractor's 
choice  of  subcontractors.   Four  alternatives  were  presented:   no 
restrictions,  Small  Business  Administration,  customer  approval  re- 
quired, or  higher  company  approval  required.   If  for  any  of  the  items 
a  contractor  subcontracted,  customer  and/or  higher  company  approval 


Kendall  Tau  =  -0.29  (p<.006,  N=41) .  In  this  and  all  subsequent  cor- 
relations the  performance  scale  has  been  reversed  so  that  a  positive 
correlation  is  more  easily  Interpreted. 

In  addition,  percent  effort  subcontract  and  type  of  contract  (sole 
source  vs.  competitive)  are  unrelated  suggesting  that  these  factors 
influence  performance  independently. 

2 

There  are  difficulties  inherent  in  having  to  monitor  subcontract  activ- 
ity wliich  could  result  in  cost  or  schedule  slippages  but  no  such  effects 
were  found  in  this  study. 
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was  checked  as  being  relevant  this  x>;as  called  restrictions.   Those 
cases  where  the  contractor  checked  no  restrictions  and/or  Small 
Business  Administration  as  beinf^  relevant  for  all  of  the  items 

subcontracted  i^iere   classified  as  no  restrictions.   The  distinction 

then  is  between  complete  freedom  from  any  higher  level  control  vs. 

the  existence  (complete  or  partial)  of  some  formal  higher-level 
approval  mechanism. 

There  were  33  projects  that  had  any  subcontracting  (See  Table  II) 
and  data  concerning  the  nature  of  restrictions  on  choice  of  subcon- 
tractors were  available  for  24  of  these.   When  technical  performance 
is  correlated  with  the  nature  of  restrictions,  the  following  results 
emerge: 

Fean  Performance 
No  Restrictions  (N=12)         4.2 
Restrictions  (N=12)  3.0 

From  this  it  can  be  concluded  that  the  existence  of  some  formal 

2 
higher-level  approval  mechanism  is  strongly  associated  with  better 

technical  performance.   Furthermore,  \/hen  the  relationship  between 


Tlie  assumption  liere  is  that  reasons  other  than  technical  ability 
underlay  the  sources  suggested  by  tlio  Small  Business  Administration. 

2  I'ann-Lliitney  U-Test  p<.02 


-    12   - 


"percent   sul)Contrac.t"    nnd   "nature   of    restrictions"    is    examined, 
the   following   results    emerge: 


Pubcontrnct        Fubcontract 


No 
Restriction? 

7 

5 

Restrictions 

5 

7 

X     =0.17 
P<.70 


In  other  v;ords,  a  high  degree  of  subcontract  activity   and  a 
lack  of  any  formal  revie\.'  in  the  choice  of  subcontractors  appear  to  ./ 
have  independent^  and  detrimental  effects  on  tec'mical  performance. 

Concerning  choice  of  subcontractors,  it  may  be  that  within  the 
customer  organization,  tiie  copnlzmt  individuals  have  liad  more  ex- 
perience with  a  larger  number  of  subcontractors  and  are  therefore 
in  a  better  position  to  judge  a  subcontractor's  capacity  to  do  a 
superior  technical  job.   In  the  cases  where  there  were  no  restrictions 
or  only  the  marginal  constraints  im.posed  by  the  Small  Business  Admin- 
istration, other  less  relevant  criteria  may  have  entered  into  the 
choice  of  subcontractors  to  the  detriment  of  technical  performance. 

Initial  Estimates 

The  next  factor  to  be  considered  were  certain  characteristics 
of  the  initial  cost,  schedule,  and  technical  estimates  made  by  the 


Due  to  the  reduced  sample  size,  the  relationship  between  "%  subcontract" 
and  technical  performance  was  recomputed  and  the  results  were  the  same 
as  those  presented  earlier. 

Both  of  these  factors  are  unrelated  to  the  type  of  contract  (sole  source 
vs.  competitive). 
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contractor  proposal  team.   The  rationale  here  was  that  using  a  par- 
ticular scheme,  such  as  PERT,  to  schedule  a  project  initially  might 
force  a  closer  look  at  technical  objectives  and  thereby  influence 
final  technical  performance. 

No  significant  differences  in  technical  performance  were  found 
to  be  associated  with  the  possession  of  additional  technical  infor- 
mation (other  than  that  included  in  the  RFP) ,  the  using  of  PERT  or 
PERT  cost  vs.  other  methods  of  scheduling  the  project,  or  the  level 
of  detail  used  in  making  initial  cost  and  schedule  estimates. 

The  only  question  analyzed  thus  far  which  very  strongly  dif- 
ferentiated in  terms  of  technical  performance  was  the  following:   Did 
you  include  a  safety  factor  in  your  initial  cost  and  schedule  estimates? 

For  15  projects  where  the  project  manager  (who  was  in  virtually  all 
cases  also  the  proposal  manager)  answered  yes  to  this  question,  the 

technical  performance  was  2 .80 .   For  the  20  projects  in  which  the  re- 
sponse was  no,  technical  performance  was  4.0.   The  inclusion  of  a 

2  3 

safety  factor  is  significantly  related  to  superior  technical  performance. 

In  terms  of  the  arbitrary  definition  of  "highly  successful"  projects 

discussed  earlier,  56%  of  the  Yes  group  and  30%  of  the  No  group  would 

be  classified  as  highly  successful. 


More  than  6  milestones  was  considered  "specific"  while  fewer  than 
6  milestones,  total  effort,  and  annual  effort  were  considered 
"not  specific." 

2 

Mann-lv/hitney  U-Test,    p<.01 

3 

As   might  be  expected   the    inclusion  of   a  safety   factor   is   strongly 

associated    (x2=5.4   p^.02)   with    the  non-occurrance   of   an  overrun. 
Again,    technical   performance   and   cost/schedule   performance    are   un- 
related. 
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It  may  be  that  an  organization  which  was  initially  very  op- 
timistic about  its  technical  proposal  felt  it  could  afford  to  build 
a  safety  factor  into  its  cost  and  schedule  estimates.   The  high 
resulting  technical  performance  could,  in  a  sense,  mean  that  initial 
optimism  was  realistic.   This  possibility  is  mitigated  somewhat, 
however,"  by  the  fact  that  no  relationship  was  found  between  the  in- 
clusion of  a  safety  factor  and  being  a  sole  source  contractor. 

From  another  point  of  viev/,  building  a  safety  factor  into 
initial  cost  and  schedule  estimates  may  permit  the  contractor  (par- 
ticularly the  project  manager)  to  focus  more  of  his  efforts  on  the 

technical  job  at  hand.   Since  he  knows  he  has  a  cushion,  he  no 
longer  has  to  worry  on  a  day-to-day  basis  about  running  out  of  funds. 
Here,  timing  may  be  a  very  important  factor.   A  cushion  or 

backlog  of  dollars  may  be  critically  important  in  the  early  phases 

2 
of  a  project  effort.    On  the  other  hand,  extra  time  and  money  across 

the  life  of  a  project  may  stimulate  the  contractor  to  do  that  "little 
bit  extra"  to  transform  an  acceptable  job  into  an  excellent  job. 

Project  Team  Characteristics 

One  of  the  most  critical  decisions  an  organization  makes  before 
beginning  work  on  a  project  concerns  the  selection  of  project  person- 
nel.  Project  managers  were  asked  the  following  questions: 


%  Subcontract  and  nature  of  restrictions  were  also  found  to  be  un- 
related to  the  inclusion  of  a  safety  factor. 

2 

See  Roberts  (  1964  )  for  a  fuller  discussion  of  a  similar  point. 
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How  many  key  people  who  worked  on  this  project  were  chosen 
primarily  on  the  basis  of  each  of  the  following  criteria;  avail- 
ability, general  capability  in  the  area,  past  association  with  a 
similar  task,  or  specialized  talent? 

What  percent  of  the  remaining  project  personnel  were  chosen 
on  the  basis  of  each  of  these  same  criteria? 

The  term  key  people  was  specifically  left  undefined  since  it 
was  believed  to  be  a  term  with  which  project  managers  were  very 
familar.   ^^Jhen  the  interviewer  was  asked  what  was  meant  by  the 
term  (very  few  project  managers  asked  the  question)  the  response 
was:   Someone  who  can  really  make  a  project  go;  if  he  (a  key  man) 
were  transferred,  the  project  would  feel  his  loss. 

Summing  across  the  four  criteria  yields  the  project  manager's 
estimate  of  the  absolute  number  of  key  people  who  worked  on  the 

project.   The  range  was  2-60  with  a  median  of  10  key  people.    A 

2 
strong  positive  correlation  was  found  between  //  key  people  and 

technical  performance;  the  more  key  people  assigned  to  a  project, 

the  better  the  project  did  technically.   As  one  might  expect,  the 

#  of  key  people  is  positivily  correlated  with  the  //  of  full-time 

professionals  working  on  the  project.   In  addition,  within  this 

4 
subsample  of  the  total  sample,  a  moderate  relationship  exists  be- 

t\r7een  technical  performance  and  the  //  full-time  professionals. 


The  project  with  60  key  people  had  over  200  full-time  professionals 
^  Kendall  T=0.31,  N=3A,  p<.01 

Kendall  T=0.66,  N=34,  P<.01 
^  Kendall  T=0.24,  N=34,  n<^  .07 
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These  relationships  can  be  summarized  in  a  three  variable  model: 


(3) 
#  KEY  rropLE  4- 


+  .66 


+  .3 


(2) 
//  FULL  TIME  PROFESSIONALS 


+  .2A 


TECHNICAL  PERFORMANCE 
(1) 


Utilizing  the  method  of  partial  correlations,  it  can  be  shown 
that  the  correlation  between  //  full-time  professionals  and  technical 
performance  disappears  when  one  controls  for  (eliminates  the  effect 
of)  the  number  of  key  people.    The  correlation  between  #  key  people 

and  performance  drops  only  moderately  when  one  controls  for  //  full- 

2 
time  professionals.    In  other  words,  the  number  of  full-time  pro- 
fessionals influences  technical  performance  indirectly  by  increasing 
the  likelihood  that  a  large  #  of  key  people  will  be  X'jorking  on  the 

project  and  it  is  the  #  of  key  people  which  directly  affects  tech- 

3 
nical  performance. 

Turning  now  to  the  criteria  for  selection  of  key  people,  no  re- 
lationships are  found  between  technical  performance  and  the  number  of 
key  people  chosen  primarily  on  the  basis  of  availability,  general 


T  1,2.3  =  0.04,  88%  reduction 
T  1,3.2  =  0.21,  32%  reduction 


No  relationship  is  found  between  performance  and  the  percent  of 
full-time  professionals  considered  to  be  key  people. 
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capability  and  past  association.    The  number  of  key  people  chosen 

by  specialized  talent  v/as  however,  significantly  related  to  tech- 

2  3 

nical  perf orniance   as  well  as  to  the  absolute  number  of  key  people. 

These  relationships  can  be  summarized  in  a  three  variable  model: 


(2)            .48  (1) 

//  SPEC.  TALENT  #  KEY  PEOPLE 


+  .27\  /    +.31 


PERFORMANCE 
(3) 


Again,  using  the  method  of  partial  correlations,  it  can  be  seen  that 

neither  of  the  direct  correlations  with  performance  disappears  com- 

4 
pletely  as  v/as  the  case  in  the  prior  analysis.    It  v/ould  appear 

that  the  #  of  key  people  and  #  of  key  people  chosen  by  specialized 
talent  have  a  moderate  effect  upon  technical  performance  independent 
of  the  relationship  between  them. 


For  11  of  34  projects,  none  of  the  key  people  were  chosen  on  the  basis 
of  past  association.   For  those  in  which  any  key  people  were  chosen 
on  this  basis,  the  correlation  between  //  key  people  and  performance 
is  T=0.32  (p{.02,  N=23). 

^  Kendall  T  =  0.27,  N=34,  p<.03 

^  Kendall  T  =  0.48,  N=34,  P<.01 

^  T  1,3.2  =  0.22;  T  2,3.1  =  0.14 

None  of  the  "key  people"  variables  are  related  to  type  of  contract 
(sole  source  vs.  competitive),  %  of  subcontract,  restrictions  on 
choice  of  subcontractors,  or  the  inclusion  of  a  safety  factor  in 
cost/schedule  estimates. 
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Concerning  the  criteria  of  selection  of  other  project  per- 
sonnel, no  relationship  is  found  between  technical  performance 
and  the  percent  of  other  project  personnel  chosen  on  the  basis  of 
availability,  specialized  talent,  or  general  capability.   The 
only  suggestion  of  a  relationship  appears  beti^jeen  percent  chosen 
by  past  association  and  technical  performance.    Analysis  of 
more  complex  relationships  is  necessary  before  anything  further 
can  be  said  in  this  area.   It  may  be,  for  example,  that  //  of  full- 
time  professionals  available  (and  used)  increases  the  liklihood 
that  a  large  number  of  key  people  will  be  available  Vi^hich  in  turn 

makes  selection  by  specialized  talent  more  feasible  leading  to 

2 
higher  technical  performance. 


^   Kendall  T  =  0.22,  N=34,  p<.06 


2 

Data  concerning  the  project  manager  (education,  experience,  etc.) 

have  been  eliminated  from  consideration  here  and  will  form  the 

subject  of  a  future  paper. 
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Summary 

The  findings  presented  in  this  paper  can  be  summarized  graphically 
in  the  following  manner: 


TYPE  OF 
CONTRACT 


Sole  Source  Contracts 
Do  Better  Technically 


PERCENT  EFFORT 
SUBCONTRACTED 


The  Less  Effort  Subcontracted. 

The  Better  is  Technical  Performance" 


\k 


1 


OTOICE  OF 
SUBCONTRACTORS 


"No  Restrictions"  is  Associated 

With  Poorer  Technical  Performance       ^ 


TEQIKICAL 
PERFORMANCE 


INITIAL 
ESTIMATES 


Inclusion  of  a  Safety  Factor  is 

Associated  with  High  Technical  Performance 


t 


KEY   PEOPLE-NUMBER  AND 
CRITERIA   FOR   SELECTION 


A  High  Absolute   //   of   Key 
People   and  a  Large  // 
Chosen   on    the  Basis    of 
Specialized   Talent   are 
Associated  with   High 
Technical   Performance 
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As  pointed  out  earlier,  data  are  not  presently  available  to 
explain  why,  for  example,  a  particular  contract  was  awarded  on  a 
sole  source  basis  or  what  it  is  about  having  to  subcontract  a 
large  percent  of  the  total  effort  whicli  leads  to  poorer  technical 
performance. 

Answers  to  these  questions  could  be  extremely  important  when 
one  considers  the  time  frame  of  the  variables  discussed.   All  of 
them  represent  the  result  of  decisions  made  by  the  government  and /or 
the  contractor  prior  to  the  start  of  any  full  scale  work  on  the 
project.   A  better  understanding  of  why  such  decisions  seem  to  ef- 
fect technical  performance  could,  for  example,  lead  to  revisions 
in  contractor  selection  procedures.   Tn  the  case  of  the  finding  con- 
cerning the  value  of  including  a  safety  factor  in  initial  estimates, 
changes  m.ight  be  made  in  the  manner  ("hoi^r  much  when")  of  funding 
project  efforts. 

The  focus  in  this  paper  has  been  on  the  relationship  betvreen 
certain  inputs  and  outputs  -  the  project  has  been  viewed  as  a  black 
box.   This  is  not  meant  to  imply  that  other  factors  which  occur 
during  the  life  of  a  project  are  unimportant  v/ith  respect  to  technical 
performance.    What  is  suggested  is  that  more  attention  could  be  paid 
to  the  impact  of  initial  conditions  over  which  both  the  government 
and  the  buyer  may  liave  considerably  more  control. 


Future  papers  will  relate  technical  performance  to  such  factors  as 
the  project  manager  (education,  past  experience,  patterns  of  effort 
allocation),  project  priority,  customer  monitoring  of  project 
(amount  and  type),  and  other  factors  which  occur  during  the  life 
of  a  project. 
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